🧠 Data Led Growth project | Freshworks

🧠

Data Led Growth project | Freshworks

Picking the Right Experiment

To improve the efficiency of marketing spend, the performance marketing team at Freshworks employs the following tactics:

Audience Segmentation
Data-Driven Optimizations
A/B Testing
Adaptive Budget Allocations
Conversion Rate Optimization

Points 1-4 form the foundation of our performance marketing efforts. Conversion Rate Optimization (CRO) is often something that gets overlooked but if proven successful can amplify results within existing budgets by generating incremental volumes

Overview

Performance marketing teams at Freshworks are split as follows:
- Paid Marketing - Generating paid traffic via Google, Bing, Paid Social and Marketplaces
- Organic Marketing - Generating organic traffic via ranking content on search
- Affiliate Marketing - Generating affiliate traffic by partnering with affiliate partners
- Referral - Generating traffic from internal (website) & external referral sources
While these teams own individual OKRs based on their surface area, there is a channel that is often ignored as uncontrollable - DIRECT
- Direct traffic according to SEMRUSH is the traffic that lands by typing your URL directly into their browser
- Users coming from direct traffic can be assumed as people who are either brand-aware aware / product-aware

Problem Statement

The current setup of the Freshdesk homepage presents a challenge: although direct users, who constitute 30% of traffic and ARR, frequently return and show a high propensity to convert, our messaging lacks distinctiveness and personalization for this audience segment. Presently, both new and returning users encounter identical H1 text upon arrival, hindering our ability to optimize conversion rates effectively.

Proposed Solution

To capitalize on the returning user segment, we aim to customize the H1 text to encourage action, such as signing up for a demo. This personalized messaging serves as a remarketing strategy.

Hypothesis

If we customize the first fold with customized H1 & imagery for DIRECT returning users

then our web conversion for direct users will increase by 10%

because we are leveraging the familiarity of returning users to drive higher engagement

Goal

Our goal from this experiment is to improve the yield from returning DIRECT users

As these returning direct users constitute approximately 40% of all direct users, a successful experiment resulting in a 10% increase in WCR for this segment would generate an incremental lift of approximately $400K in ARR from these users.

Success Metrics

Success in this experiment is defined by
- Primary Success Metric - Lift in Web Conversion Rate
- - Best Case Scenario - Increase in WCR by 10%
  - Worst Case Scenario - Increase in WCR by 1-2% (flat)
  - - In this scenario, given the WCR did not decrease we would like to re-run the experiment by optimizing H1 & Imagery
- Secondary Success Metrics - Lift in Visits to Signup / Demo Page
- Health Metrics - Exit / Bounce Rates

Experiment Design

What are we testing?

We are testing the first fold of the Freshdesk homepage which remains static for every channel/audience that lands on this page.
The first fold can be capitalized by personalizing the H1 text and imagery on the right to improve web conversion rates
Funnel Data - From GA4

Journey	Users	WCR%
Users landing on Freshdesk Homepage (all channels)	1.4M	1.7%
Users landing on Freshdesk Homepage (via direct)	0.45M	1.9%
Returning Users landing on the Freshdesk Homepage (via direct)	0.18M	2.1%

There is an opportunity to improve the conversion rate of returning users by personalizing the H1 and offering a product tour

Variation Design

Following changes will be made to the variant

Variant H1 text and subtext to be personalized for returning users
The video on the right will be replaced with a product tour
1. A product tour is an interactive demo of the product that explains the capabilities in 10-12 steps
Other elements besides the first fold (H1 & subtext and product tour) will remain the same as the control

Control

Variant

Audience & Sample Size

Audience
- Our audience for this test is all returning users from the direct channel landing on the Freshdesk homepage
Sample Size
- Using the Optimizely A/B test sample size calculator and adding inputs on the current baseline conversion rate, MDE, and statistical significance (95%) we get a sample size of 86,000 users
Duration of Test
- June 1 - July 30 or till we are statistically significant (95%), whichever is earlier

Implementation & A/A Test

Platform - Optimizely Web Experimentation
A/A test -
- Connect GA4 to Optimizely to measure the success metrics mentioned above (Freshdesk signup conversion rate, visits to the signup page, bounce/exit rates)
- Run the A/A test by keeping the control and variant the same and measuring the metrics defined above
- Results generated were the same across the control and variant indicating experiment setup is valid

Did not have access to test this on Optimizely, hence explanatory data is added below

Post Experiment

SAMPLE DATA ADDED BELOW - Did not have access to test this, hence explanatory data is added below

Experiment Result

After running the experiment until statistical significance we notice that the conversion rate of variant:
- Signup to Freshdesk increased by +12.17% (95% SS)
- Visiting Signup Page increased by +6.6% (95% SS)
- Exit rate remained stable across control and variant
Based on the results, the experiment can be stopped and declared as SUCCESS

Release Decision

SCALE the experiment since it was successful with statistical significance

Learnings

Leveraging user familiarity significantly boosts engagement. Prioritizing personalized experiences and understanding user behavior is crucial for driving conversion improvement.

Next Steps

As the experiment was successful and we decided to scale this experiment, we need the following next steps to push this experiment into production:

Since the copy & designs are ready we need a set this as a personalization experiment for 100% of returning direct users on Optimizely - ETA is 3-4 days
To do so, we raise a ticket with the web team and use the Optimizely tool's results page to monitor results on 100% audience

Stakeholder Management

To get buy-in for this experiment we would:

	One	Two	Three	Four
Whom to communicate	Manager (Growth Marketing)	Growth Marketing team	Web Team	Web Team & Growth Marketing team
When to communicate	During the Hypothesis building stage	After scoping the doc	Pre Launch	Post Launch
What to communicate	Hypothesis & expected result	Doc to negotiate & align on next steps	Action Items & Next Steps	Experiment Results, Learnings & Actions

One

Two

Three

Four

Whom to communicate

Manager (Growth Marketing)

Growth Marketing team

Web Team

Web Team & Growth Marketing team

When to communicate

During the Hypothesis building stage

After scoping the doc